Combining List Experiments and the Network Scale Up Method

Gustavo Díaz
McMaster University

@gusvalo

Verónica Pérez Bentancur
Universidad de la República
@veroperezben

Ines Fynn
Pontificia Universidad Católica de Chile

@ifynn_

Lucía Tiscornia
University College Dublin

@tiscornia21

   

Slides: gustavodiaz.org/talk

Background

  • Social scientists care about sensitive issues

  • Asking about them directly leads to misreporting

  • Solution: Indirect questioning techniques

  • List experiments popular in political science

Example

Now I am going to read you things that make people angry or upset

Example

After I read them all, just tell me HOW MANY of them upset you

Example

I don’t want to know which ones, just tell me HOW MANY

Control group

  1. The federal government increasing tax on gasoline
  2. Professional athletes getting million dollar contracts
  3. Large corporations polluting the environment

Example

I don’t want to know which ones, just tell me HOW MANY

Treatment group

  1. The federal government increasing tax on gasoline
  2. Professional athletes getting million dollar contracts
  3. Large corporations polluting the environment

Example

I don’t want to know which ones, just tell me HOW MANY

Treatment group

  1. The federal government increasing tax on gasoline
  2. Professional athletes getting million dollar contracts
  3. Large corporations polluting the environment
  4. A black family moving next door

Reduce bias but increase variance

Reduce bias but increase variance

Reduce bias but increase variance

Reduce bias but increase variance

Ways to reduce variance

Ways to reduce variance

Combined estimator

  • Logic: You don’t need a list experiment for those who openly confess to the sensitive item

\[ \hat{\mu} = \overline{Y} - (1 - \overline{Y}) (\overline{V}_{1,0} - \overline{V}_{0,0}) \]

  • \(\overline{Y}\): Proportion confess in direct question

  • \((\overline{V}_{1,0} - \overline{V}_{0,0})\): List experiment estimate among not confessing

Problem

  • Can’t always include direct questions

  • Combining with other indirect questions needs altered designs or extra modeling assumptions (e.g. Blair, Imai, and Lyall 2014)

What to do?

  • This project: Replace direct questions with Network scale-up questions!

Network Scale-Up Method (NSUM)

How many people do you know,

Network Scale-Up Method (NSUM)

How many people do you know, who also know you,

Network Scale-Up Method (NSUM)

How many people do you know, who also know you, with whom you have interacted in the last year

Network Scale-Up Method (NSUM)

How many people do you know, who also know you, with whom you have interacted in the last year in person, by phone, or any other channel.

  • Named Michael
  • Named Christina
  • Gave birth in the past 12 months
  • Commercial pilots
  • Have tested positive for HIV

Why NSUM?

  • Can infer individual responses to sensitive item

Assumption

If someone knows an unusually large number of people with sensitive item, then they are likely to hold the sensitive item too.

  • If true, can use NSUM responses as direct questions
  • Goal: Find individuals with large sensitive network relative to personal network

Hierarchical model

\[ \begin{align*} y_{ik} \sim \text{negative-binomial}( & \text{mean} = e^{\alpha_i + \beta_k},\\ & \text{overdispersion} = \omega_k) \end{align*} \]

  • \(y_{ik}\): Degree of group \(k\) for person \(i\)

  • \(\alpha_i\): Expected degree of person \(i\) (logged)

  • \(\beta_k\): Expected degrees of group \(k\) (logged)

  • \(\omega_k\): Controls variance in propensity to know someone from group \(k\)

Hierarchical model

  • Fit with MLE in two steps (Personal network, sensitive group network)

  • Focus on standardized residuals:

\[ r_{ik} = \sqrt{y_{ik}} - \sqrt{e \alpha_i + \beta_k} \]

  • High residual: Higher exposure to sensitive item

Application

Criminal governance strategies in Uruguay

  • Low crime, but embedded

  • Even here criminal organizations replace government

  • Fieldwork: Interactions are sensitive topic

  • Goal: Document extent of exposure to criminal governance strategies (positive, negative)

Survey

  • Facebook sample of Montevideo residents (N = )
  • Four criminal governance strategies

Negative

  • Threaten neighbors
  • Evict neighbors

Positive

  • Offer jobs to neighbors
  • Make donations to neighbors